1. Field of the Invention
This invention relates to image processing techniques of a light field, and more particularly, to a photometric calibration method for eliminating the photometric distortion of images of a scene, so as to improve the quality of the light field formed by the images.
2. Description of Related Art
Light field acquisition is of fundamental importance among all aspects of computational photography. A complete 4D light field contains most visual information of a scene and allows various photographic effects to be generated in a physically correct way. The light field is a function that describes the amount of light traveling in every direction through every point in free space.
There are several ways to capture the 4D light field by a light field camera. The simplest method involves using a single moving camera whose position for each exposure is established by a camera gantry, possibly estimated by a structure-from-motion (SMF) algorithm. This method is slow because the position of the camera or the object has to be altered before each exposure and only works well in a controlled environment. The second method captures the dynamic light field by putting numerous cameras into a camera array and each camera can capture a subset of the light field at video-rate. The method involves simultaneously capturing the full 4D dataset by the camera array, which is cumbersome and expensive. The third method involves inserting additional optical elements, such as a microlens array or masks in the camera to avoid the angular integration of the light field.
Nevertheless, the light field captured by the existing light field cameras suffers from a common photometric distortion. The light field images corresponding to the boundary of a aperture of the camera appear very different to that corresponding to the center of the aperture. For example, the vignetting effect that is attributed to the cosine-fourth falloff, the blocking of the lens diaphragm or the hood, lens or light sensor aberrations, etc. A photometric distortion in the image captured by traditional cameras exhibits radially diminishing intensity from the center of the image. In the case of 4D light field, the photometric distortion becomes so complex that the conventional models for representing the distortion in the light field become inadequate. However, the photometric distortion in the 4D light field captured by the light field camera must be eliminated; otherwise, it may obstruct view interpolation and depth estimation and thereby render the captured images of the scene less effective. Existing photometric calibration methods generally make two assumptions. First, the scene points have multiple registered observations with different levels of distortions. Second, a vignetting center is valid in most traditional camera, where the optics and the sensors are symmetric along the optical path. Some recent methods remove the first assumption be exploiting the edge and gradient priors in natural images, but the second assumption is still needed. Both assumptions are inappropriate for the light field images because the registration of the light field images taken from different view points requires an accurate per-pixel disparity map that is difficult to obtain from the distorted inputs, and, in each light field image, the vignetting center and other parameters of the vignetting function are image-dependent and coupled. Therefore, estimating the parameters of the vignetting function is an under-determined nonlinear problem.
Accordingly, it is imperative for the light field camera to employ a method of photometric calibration so as to solve the above problems.